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Abstract: We study the following Schrodinger-Poisson system 



f -Au + (1 + fJ,g(x))u + \4>{x)u = |u| p_ V x G M 3 , 
(P\) < _A0 = n 2 , lim <j>{x) = 0, 



[x[— H"DO 



where A, p are positive parameters, p G (1, 5), g{x) G L°°(IR 3 ) is nonnegative 
and g(x) = on a bounded domain in M 3 , g(x) ^ For fi = 0, (Pa) was 

studied by Ruiz [J. Funct. Anal., 237(2006), 655-674]. However, if p ^ and 
g(x) is not radially symmetric, it is unknown whether (Pa) has a nontrivial 
solution for p G (1,2). In this paper, combining domain approximation and 
priori estimate we establish the boundedness and compactness of a (PS) 
sequence, then we prove for p > large that (Pa) with p G (1,2) has a 
ground state if A > small and that (Pa) for p G [3,5) has a nontrivial 
solution for all A > 0. Moreover, some behaviors of the solutions of (Pa) as 
A — ► 0, (j, — > +oo and \x\ — > +oo are also discussed. 

1 Introduction 

In this paper, we are concerned with the existence of nontrivial solutions of 
the following Schrodinger-Poisson system 

( -Au + V^(x)u + X(j)(x)u = \u\ v ~ l u, x G K 3 , 



where V^(x) = 1 + ng(x), A and \i are positive parameters, p G (1,5) and 
g(x) satisfies 
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(d) g(x) > 0, g(x) G L°°(M 3 ). 

(G2) &o = {x G M 3 : g(x) = 0} is bounded and has nonempty interior. 
(G3) liminf g(x) = 1. (Here 1 can be replaced by any positive number) 

\x\— >oo 

Problem (jl.ip arisen in quantum mechanics, which is related to the study 
of nonlinear Schrodinger equation for a particle in an electromagnetic field 
or the Hatree-Fock equation, etc., see [6J [TOj [T2], [T7] and the references 
therein. If \x = 0, the existence of solutions of problem (jl.ip has been 
discussed for different ranges of p, for examples, [9] [TO] [12] for p G [3,5), 
0] for p G (2,5), [TO] for p G [2,3) and [2] [IT] [3] for p G (1,5) or general 
nonlinearity, etc. For some Ao > 0, it was proved in [TO] that (jl.ip with 
/i = and p£ (1,2] has no nontrivial solution if A ^ Ao- Ruiz in [T7] proved 
that Ao = 7 and that (jl.ip has solution if A > small enough. Some recent 
results in this direction was summarized in [Tj. However, if \i 7^ 0, there are 
only a few results on (jl.lj) for p G (2, 5) and some special potential functions, 
such as H [2H [201 EI]. In [J, a ground state of CO]) with p G (3,5) was 
obtained if V^(x) satisfies 

(Vi) Voo = lim Vfj,{y) ^ V^(x) a.e. in x G M 3 , and the strict inequality 

\y\-*+oo 

holds on a positive measure set. 

It was also proved in [22J that (jl.lj) still has a ground state for p G (2, 3] if 
(Vi) holds and V^x) is weakly differentiable function such that 

(V 2 ) (VV^(x),x) G L°°(M 3 ) U L 3 / 2 (M 3 ) and 2V ll (x) + (VV^(x),x) ^ a.e. 
x G M 3 , where (•, •) is the usual inner product in M 3 . 

If — Au is replaced by -eAu in (jl.ip . the semiclassical states for this kind of 
problem was discussed very recently in [TOJEI] for special V^x), and in [H] 
for fi = 0, etc. Problem (jl.ip with asymptotically linear nonlinearity was 
considered in [20J. To the authors' knowledge, it seems still open if there 
exists a solution to problem (jl.ip with \x 7^ and p G (1,2). It is known 
that (Vi) is crucial in using concentration compactness principle if V^{x) 
is not radially symmetric, (V2) is important in getting the boundedenss of 
a (PS) sequence. In this paper, we assume neither the conditions of (Vi) 
and (V2) nor that the potential V^(x) is radially symmetric. These lead 
to several difficulties in using variational method to get solutions of (jl.ip . 
The first is to prove that a (PS) sequence is bounded in ^(M 3 ), which is 
known to be hard for p G (1,2) because of the presence of the so called 
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nonlocal term "</>(x)u" in (jl.ip . If fx = 0, we can overcome this difficulty by 
using the decay property of functions in H^(M. 3 ) [T7], but this trick does not 
work when V fJi (x) is not radially symmetric. Once we get a bounded (PS) 
sequence, then we have to find a new way to prove that the (PS) sequence 
converges to a solution because the concentration compactness principle 
cannot be used in our case due to the lack of conditions (Vi) etc. In this 
paper, we first consider the problem (jl.ip on = {i £ I 3 : \x\ < k} for 
each k £ N, and we can easily get a solution Uk of (jl.ip in H^B^) since the 
Sobolev embedding H^B^) L q {Bk) for q £ (1,6) is compact. Then, we 
establish a uniform priori estimate of L°° norm of U)~. Finally, by adapting 
some techniques used in [20], we prove that {u k } converges to a nontrivial 
solution of (jl.ip . Moreover, a ground state of (jl.ip for p £ (1,2) is also 
proved in Section 6. However, our methods seem not useful for p £ [2,3). 

Notations: Throughout this paper, for k £ N, we denote B^ = {x £ 
R 3 : |x| < k} and define 

C^(M 3 ) = {u£ C^IR 3 ) : u £ L°°(1R 3 ) and |Vu| £ L°°(M 3 )}, (1.2) 
Sy = {u £ ^ 1,2 (IR 3 ) : ( VJx)u 2 dx < +oo}, (1.3) 

S k = {u £ ®l> 2 {B k ) : / ^(a;)u 2 dx < +oo}. (1.4) 
JB k 

Clearly, Sy and S k are Hilbert spaces, their scalar products are given by 
(u, v)g v = I VuVv + V^{x)uv dx, for any u,v £ Sy, (1-5) 

(u,v)k = / VuVv + V^{x)uvdx, for any u, v £ 3> k . (1-6) 
The norms of 3>y and ^ are introduced by 

IMliv = ( n ' n >5V> IMIfc = ( w > n )fc- (1-7) 

Clearly, [| • \\@ v is an equivalent norm of ^T 1 (1R 3 ). For p £ [1, +oo], we denote 
the usual norm of L P (Q) by | • \lp(u)i an d simply by | • | p if = R 3 . 
Moreover, for any tie^, the extension of u on M 3 is defined by 

u(x) = u(x) if x £ B k , u(x) = if x £ B%. (1.8) 
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For u G let <f>u be the unique solution of —Ac/) = u 2 in S$ 1,2 (M. 3 ) [Lemma 
12.11 below], define 

I(u) = -[ \Vu\ 2 + V(x)u 2 dx + ^ [ 4> u {x)u 2 dx — f \u\ p+1 {x)dx. 

2 Jr3 4J K 3 p+l J R 3 

(1.9) 

By pll Proposition 2.1], I £ C^^R) and for any 

we have 

l'(u)ip= / VuVip + V(x)mpdx + A / 4> u wpdx — / |-u| p_1 uc/j(i2;. (1.10) 

Moreover, if -u G £Fy and I'(u)<p = for all </> G i^y, Lemma 2.4 of [TO] 
showed that u and </> u (simply by <j), sometimes) satisfy (jl.ip in the weak 
sense. 

Definition 1.1 A pairs (u,4>) G 3)y x £F 1,2 (IR 3 ) is said to be a (weak) solu- 
tion of if 

I'(u)(p = for all ip G @v- (1-H) 

For the sake of simplicity, in many cases we just say u G Sh/, instead of 
(u,(f) u ) G 3>v x ^ 1,2 (K 3 ), is a weak solution of (fTTTj) . 
We end this section by giving our main results. 

Theorem 1.1 Let p G (1,2) and (Gi) to (G3) hold. Then there exists A* G 

2 2p-l P 

(0, +00) and C Pi a = 2p~ 2 (2 — p) p- 1 [p(p — 1) A - 1 J 2 ~ p such that problem 
hl.l\) has one nontrivial solution (u,<p) G Sh/ x ^ 1,2 (M 3 ) for any A G (0, A*) 
and [i > [ii = C%~^ — 1. Moreover, 

I(u)£[a,c\], \\u\\cjf v + I V0| 2 < Mfj, := M(p, X,fi), and 

\u(x)\ ^ C Pt \, < <p(x) ^ := C(p,X,fi) a.e. in x G M 3 , 
where a,c\ > are independent of \x, and is decreasing in fi. 

Remark 1.1 We claim that A* ^ c(p) = \{p — 1) 2 (2 — p) 2 ~ . In fact, 
similar to the proof of \17\ Theorem J^.l], we see that fZHP has no any 
nontrivial solution in Q!y x f^ 1,2 (IR 3 ) if A ^ c(p). Moreover, it is not difficult 
to check that C Pj \ > 1 if X £ (0,c(p)). Hence /xi > in Theorem \l.t\ 

By Theorem 11.11 and Remark \\.\\ we know that the existence of solutions 
of (jl.ip for p G (1,2) depends heavily on the parameter A. However, if 
p G [3, 5), the situation is quite different and we have the following theorem. 
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Theorem 1.2 Let p G [3,5) and (Gi) to (G3) ao/d. Then, for any A > 
0, i/tere exist positive constants Mq := Mo(A), Mi := M\{\) and M := 
M(A), which are independent of /x and nondecreasing in A > 0, snc/t i/iai 
problem has one nontrivial solution (u,4>) G fIV x ^ 1,2 (IR 3 ) if fi > 

H2 = max{0, M (A) P_1 — 1} ; and (n, 0) satisfies 

\u(x)\ sC M and < </>(x) ^ M x a.e. in x £ R 3 , \\u\\y v + |V^| 2 < M. 

Moreover, there exist a > and > 0, independent of fj,, such that I\(u) G 
[a,c\] where a is also independent of X > 0. 

The following three theorems describe how the solutions of (11. IB behave 
when A — > 0, /U — > +00 and |x| — > +00, respectively. 

Theorem 1.3 Under the assumptions of Theorem \1.2l For each A > and 
H > fj,2, let u\ denote the solution of obtained by Theorem Then 
there exists a solution uq G 3>y \ {0} for U.l\) with A = such that, along a 
subsequence, 

u\ — ► no strongly in @>y as A — > 0. 

Theorem 1.4 For eac/i fi > iarae, iei n^ 6e a solution of hl.l\) obtained 
by Theorem li.il or I HH Then, there is u G -ff 1 (]R 3 ) loii/i n(x) = a.e. in 
x G M 3 \ Slo and n(x) ^ in snc/i iaai, passing to a subsequence, 

—* n in id^IR 3 ) as /U — > +00. 

Moreover, if dilo is Lipschitz continuous, then u G id^^o) an< ^ ^ * s a weaA; 
solution of the following problem 

-Au + u + ±(u 2 *-^)u = \u\ p - 1 u, ie!lo, ^ 12 . 

n(x) = 0, x G <9f2o- 

Theorem 1.5 For each A G (0, A*), let be the solutions of ( ti.il) obtained 
by Theorem \l.l\ or \l. l A If u > uo = max{3Cp^ — 3, 3Mq _1 — 3}, toen there 
exist A > and i?o > which are independent of u snc/i i/iaf 

n M (x) < A\x\~^e~ for \x\ > R and a > n . 

Theorem 1.6 For A* and ui given by Theorem \l.l\ let p G (1,2) and (Gi) 
to (G3) ao£d. Then problem (fj.J)) aas a ground state (u,(p) G x ^ 1,2 (IR 3 ) 
/or eac/t A G (0, A*) and fj, > fj,%. 
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2 Nontrivial solution for (11.11) on Bk 

For any k G N, we consider the following problem 

—Am + V^(x)u + X(j)(x)u = \u\ p ~ 1 u, x G B k , 
-A0 = n 2 , ' xeB k , (2.1) 

u(x) = 4>{x) = 0, x G 

For u £ S>k and (p := (p u £ &Q 2 (B k ), define 

:= = J / |Vn| 2 +F M (x)n 2 dx+^ / <p u {x)u 2 dx l — [ \u\ p+1 dx, 

2 J Bk ^ JB k P + 1 

(2.2) 

and 4 G C 1 ^,^), {u,4> u ) e & k X &l' 2 (B k ) is a weak solution of (pTlj) 
if and only if n is a nonzero critical point of I k (u). The main aim of this 
section is to prove that 

Theorem 2.1 Let p G (l,2)u[3, 5) and (Gi) to (G 3 ) /ioW. Then there exist 



A* > and ko G N snc/i i/iai, /or k > ko, problem \2. 1\) has at least one 
nontrivial solution (u k ,<p k ) G & k x ^ ,2 (B k ) with 

h(uk) = c\,k G [a, c A ] for each A G (0, A*), 

where A* = +oo i/p G [3,5) and A* G (0, +oo) if p G (1,2), a and c>, are 
positive constants independent of k and fj, ( a is also independent of X). 
Moreover, for k > k , c Xl , k ^ c X2 , k if Xi ^ A 2 . 

Before proving Theorem 12.11 we recall some results for the following 
Poisson equation on any smooth domain $7 C M 3 , O may be unbounded, 

-A(p(x) = u 2 , x G n, 

G ^ 2 (fi). (2 - 3) 

The following lemma is a result of Lax-Milgram theorem. 

Lemma 2.1 Let u G L 12 / 5 (Cl). Then $£J$ has a unique solution <f> u G 
S>q ,2 (VL) such that 

|V0 u | L2(n) < S^'M 2 ^ [ <p u u 2 (x)dx < ^Vl 4 ^ (2.4) 

w/iere So = inf{/ R3 |V«| 2 dx : n G £F 1,2 (M 3 ) and J* R3 |n| 6 dx = 1} is the 
Sobolev constant, which is independent of £1 and u. 
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Lemma 2.2 ( \T% Lemma 2.1]) For {u n } C L 12 / 5 (f2) with u n —> u strongly 
in L 12 / 5 (f2), let (j> n and <fi be the unique solutions of Ii2.3\) corresponding to 
u n and u, respectively. Then, 

<p n A <j) strongly in ^q' 2 (0) and / (\> n u 2 n dx —* \ cj)u 2 dx. 



Lemma 2.3 Let {uk} C L 12 / 5 (M 3 ) be such that u k u strongly in L 12 / 5 (]R 3 ) 
for some u G L 12 / 5 (R 3 ), and (j) k G @l' 2 {B k ) be the unique solution of 12. 3\) 
with f2 = B k and u = u k . If 4> k is the extension of 4> k on M 3 defined as 
\1.8\) . then 

4>k(x) A <j>(x) = ]- I , U ^ V \ dy strongly in ^ 1 ' 2 (R 3 ), 



4vr J R 3 \x-y 

where <j) is essentially the unique solution of A2.3\) with £1 



Proof. For u G L 12 / 5 (M 3 ), by Lemma O there is a unique <f> G S> ' 
such that 

/ V(/)V<pdx = / u 2 ipdx, for any ip G ^ 1 ' 2 (M 3 ). (2.5) 

JR3 J R 3 

Moreover, 4>{x) = ^ / K3 jj^^dy by Theorem 9.9 of [14]. By Lemma 12.11 
there is C > independent of B k and u k such that 

|V0 fc | 2 = |^ |V0 fc | 2 dx| sC C\u k \ 2 L12/5{Bk) ^ C\u k \l 2/5 , 

this implies that is bounded in f^ 1,2 (lR 3 ), since u k —> u strongly in 

L 12 / 5 (M 3 ) and {u k } is bounded in L 12 / 5 (R 3 ). Hence, there exists <f> G 

^ 1 ' 2 (M 3 ) such that 4> k 4> weakly in ^ 1 > 2 (M 3 ), that is, 

/ V4> k X7<pdx / X74>V<pdx, for any ip G ^ ll2 (R 3 ). (2.6) 

By the definition of (ft k , f|2.6[) yields 

/ V(j) k Vipdx = / V(f> k Vipdx — ► / V(j)V(pdx, for any G Cq 
7s t JM 3 Je 3 



:^°(M 3 ) 



(2.7) 
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For any ip G Co°(R 3 ), since B k — > M 3 we see that suppy? C B k for all k £ N 
large. Noting that <j) k satisfies (|2.3p with u = u k and = B k , it follows from 

k 

u k — > u that 

/ V(j) k V(pdx = / ulfdx + o(l) = / nf^dx + o(l) = / u 2 (pdx + o(l), 
is,, Js fe Jr 3 Jr 3 

(2-8) 

where and in what follows o(l) denotes the quantity which goes to as 
k — > +00. Therefore, combining (|2.7p (|2.8p and (|2.5p . we see that 

V(f)V<pdx = / n^dx = / VcpVipdx, for any 99 £ ^ ll2 (M 3 ). 

So, = 4> hi £F 1,2 (IR 3 ). Taking = and in (|2.5p and (|2.6p . respectively, 
then, 

/ u 2 4> k dx= [ V(/)V4> k dx^ [ |V0| 2 dx. (2.9) 
On the other hand, 

/ \V4> k \ 2 dx = / |V^A;| 2 dx = / u 2 k 4> k dx = / ul4> k dx. 
Jr3 is,. Js fc Jr3 

By u k —>■ u strongly in L 12 / 5 (]R 3 ), it follows from (|2.9p that 

|v4| 2 dx-^> / |V<£| 2 dz, 



this implies that <p k (f) in ^ 1 ' 2 (M 3 ). □ 

Now, we are ready to prove Theorem 12.11 For using Mountain Pass 
Theorem, the following properties for I k defined by (|2.2p are required. 



Lemma 2.4 Let p £ (1,2) U [3,5) and (Gi) to (G 2 ) hold. We have that 

(i) There are p > and a > (independent of k, A and //J suc/i i/iaf 

inf I\,k(u) ^ a > 0, /or all \ > 0,k £ N. 

ue^;||u|[fc=p 

(ii) There exist A* G (0, +00) if p £ (1,2) or A* = +00 if p £ [3,5) and 
e € £F 1,2 (R 3 ) suc/i i/tai suppe C ||e|L > p and 



Ix,k( e ) < 0; f or ^ £ (0? ^*) an d k £ ~N large, 
where e is independent of X only if p £ (1,2). 



S 



(iii) There is a constant c\ > 0, independent of k and p, such that 

c x ,k ■= inf max I\,k(l(t)) ^ c A < +00, 
7 er A te[o,i] 

where T x = {7 G C([0, 1], 9 k ) : 7 (0) = 0, || 7 (l)|| fc > p, J A ^( 7 (1)) < 0}. 
Moreover, for any fixed k G N, < a ^ c Al ^ ^ ca 2i /c «/ Ai ^ A2. 

Proof (i) For g G (2,6), let 

S = inf{ / \Vu\ 2 + u 2 dx : u G iI 1 (R 3 ) and = 1}, 

which is independent of k and For any uG^C H^B^), define tt as 
(|1.8p and tt G C i? 1 (M 3 ). Then, Sobolev embedding implies that 



/ \ur l dx = [ \u\^dx < S-^|N|£| R3) < = S-^Hnllf \ 

Since A > and (p u > 0, it follows from (|2.2p that 



/ ■ , 1 II 112 1 c- 2 ^!! IIP+1 



By p > 1, it is not difficult to see that there exist positive constants p, a 
(independent of k, A and p) such that (i) holds. 

(ii) We separate the proof into two cases: p G (1, 2) and p G [3, 5). 

If p G (1, 2), let w G C£°(1R 3 ) \ {0} with suppw C O . Since Q C B k for 
k G N large, there is /co G N such that, for all k > ko and t > 0, 

IoA^) = h.k(tw)\x=o = - \Vw\ 2 + w 2 dx — / \w\ p+1 dx. (2.10) 

2 A} P + 1 Jn 

Clearly, there is to > large enough such that e = tow satisfying Io,fc( e ) < 
and ||e||fe = iolMlHi(o ) > p, here e is independent of k, A and p. Since 

I\,k{&) = ^o,fc(e) + f Jq 4> e e 2 dx is continuous in A ^ 0, combining Jo,fc(e) < 
and Lemma [2. II we see that there exists A* > small (independent of k and 
p) such that A,fc(e) < for all A G (0, A*). 

If p G [3,5), the above proof for p G (1,2) still works but by that way 
A has to be small. However, for p G [3,5), we can prove that (ii) holds 
for all A > 0, that is A* = +00. In fact, by intOo 7^ we may assume 
that B £q {xq) = {x G M 3 : \x — xo\ < Eq} C f2o for some xq G fto an d 
£ > 0. Taking w G C^°(M 3 ) with suppw C -B eo (0) and letting w t (x) = 
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t 2 w(t(x — Xq)), then suppu>t C B £0 (xo) for t > 1, and B £Q (xo) C £lo C B k if 
fc£N large enough. Since g(x) =0 on £Iq, for t > 1, we have 



h,k(wt) = \ ( \Vw t \ 2 + w 2 dx + ^ f 4> Wt w 2 dx 3— f \w t \ p+1 d. 

2 Jn ^ Jn P + 1 Jn 

* 3 /" ,„ ,2 , * /" 2 , Xt3 f , 2 r t 2 ^ 1 f 

= — / I Viol dx + - w dx + — I <p w w dx / 

2 Jn 2 7n 4 Jn p+l Jn 



Since p > 2, 2p — 1 > 3, we see that, for each A > 0, there exists t := 
to (A) > 1 large (independent of k and p) such that 

h,k( w to) < and INollfc = -W / |Vw| 2 dx + — / w 2 (ix > p 2 . 

Hence (ii) is proved by taking e := e\ = wt (independent of k and p). 

(iii) By the result of part (i), it is obvious that ^ a > 0. 

For pG (1,2) U [3, 5), by part (ii), there always exists such that 

h,k( e ) < °- Let 7o(*) = te for * G [0> !]• Then 7o(*) G T A and 

CA.fe ^ max I\ tk (lo(t)) = max J A fe (te) = c A < +oo. 
te[o,i] *6[o,i] 

Note that suppe C and e is independent of /c and p, so c A is also in- 
dependent of k and p. For each k > ko and for any u G (12, 2j) implies 
that I\ 1: k(u) ^ I\2,k( u ) if ^ ^2) then the definition of T A yields that 
Ta 2 £ . Therefore 

c\ l k = inf max I\ uk (l(t)) < inf max h 2 ,k(l(t)) = C\ 2 ,k- □ 
7er Al t G [o,i] 7er A2 te[o,i] 

Proof of Theorem I2.lt Since Jfe(0) = 0, it follows from Lemma [2~4l 
and the mountain pass theorem that, for each k G N large and A G (0, A*), 
there exists {u n } := {itl } C &ki n = 1, 2, • • • , such that 

h(u n ) ^ c x ,k and 4(«„) in 

where f^jj* is the dual space of fi^. Hence, as n — > +oo, 

7: / |Vn n | 2 +V r / ,(2;)ti 2 l (ix+^ f <\) n u 2 n dx 3— / \u n \ p+1 dx = c Xk +o{l), 

2 iB fe 4 JB k P+ 1 JB k 

(2.11) 

= / \Vu n \ 2 + V^{x)Undx + \ I (pnU^dx - \u n \ p+1 dx, (2.12) 

JB k JB k JB k 
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/ Vu n Vip + Vfj,(x)u n cpdx + A / 4) n u n tpdx - I \u n \ p 1 u n ipdx = o(l) , 

(2.13) 

for any ij? 6 fj., where (f> n denotes the unique solution of Poisson equation 
— A(j) n = in @ k . For each k G N large, if {u n } = {u^} is bounded in @ k , 
then there exists it^ G ^ such that, passing to a subsequence, 

u n u k weakly in Qi k and strongly in L q (B k ) for g G (1,6), 

since the embedding & k C Hq^B^) <— > L q (B k ) is compact. Hence, Lemma 
[231 follows that 

/ 4> n u n dx -> / <j) Uk u k dx, where - A<f) Uk = u k in %' (B k ). 

J Bh J Bi. 



Based on these facts, it is not difficult to see that u n — > u& strongly in ^ by 
(I2.12p and (|2.13j) . Hence, Ik(u k ) = ca,/c G [a, c>] by Lemma l2T4"t and ^ 
is a weak solution of (|2.ip . Therefore, to prove Theorem 12.11 we need only 
to show that {u n } is bounded in S> k . 

If p G [3,5), then | — ^-j- ^ 0. Since I' k (u n ) — > in subtracted 
i x (l2~T2ll from (|2TTTj) which gives 

ll^nlll ^ 8c k + 2, for n large enough, (2-14) 

hence {u n } is bounded in f^,. However, if p G (1,2), the properties of 
Ik(u) change greatly. In this case, we have to use other trick to get the 
boundedness of {u n } in Ql k . By — A<j) n = v? n in ^' 2 (B k ), we have that 



\/X / \u n \ 3 dx = \/A / V^„V|n„|dx ^ - / |Vu n | 2 (ixH — / |V(/> n | 2 (fx 
JB k JB k 2 J Bfc 2 J Bk 

= o / l Vu n! 2rfx +o / ^nW 2 ^- (2.15) 

Since I' k {u n ) A in f^*, we may assume that ||-^(itn)||^* ^ 1 for n large. 
Then, by (|2.12p and ()2.15j) we see that, for n G N large, 

;j / |Vu n | 2 + ^^2; + v 7 ! / \u n \ 3 dx^ I \u n \ p+1 dx + \\u n \\ k . (2.16) 
If p G (1, 2) and the Young's inequality yields that 



\u n \ p+1 dx < 



(» + l)e. ,o 2 - p _£±i 



P+i 



r— f . „ , 2-p/3\/A\ , ,. 3^ 

V A / ti/di H — — — \B k \, by taking e = — — . 

JB k 3 \p+l p+l 
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This and (|2,16p imply that 



1„ .., „ , 2-p ( 3\/A \ 2 ~ p .„ . p 

o Pn U ^ K * H 5— — rr ^ ' for n lar § e - 

2 3 lp+U 

So, {n n } = {i4?} is bounded in ^ for each k G N large. □ 



3 L°°-norm priori estimates for solutions of prob- 
lem ( gap 

The aim of this section is to establish priori estimates of L°°-norm for solu- 
tions of (jl.ip and (12. ip . If p G (1, 2), Lemma [3. II shows that the L°°-norm of 
solutions of (jl.ip or (|2.ip is bounded above by a constant depending only on 
A and p. If p G [3, 5), similar to (|2.14j) . we know that the solution u& of (|2.ip 
is uniformly bounded in 9\. for all G N and so does in L 6 (S^), then in 
Lemma [3731 we establish an estimate of |^fc|L°°(_B fe ) by using |iife|i6 (B fe )- These 
priori estimates are important in showing that the solution Uk of (12. 1|) is uni- 
formly bounded with respect to the domain and in showing that %, the 
extension of n^ on M 3 , converges to a nontrivial solution of (jl.ip . 

Lemma 3.1 For p G (1,2) and = B k or = M 3 , Zei (it, 0) G #o(fJ) x 



(3.1) 



(O) 6e a u>eaA; solution of the following problem 

-Ait + V r (x)it + X(j){x)u = \u\ p ~ l u, x G fi, 
-A0 = u 2 , x G 0, 

uAere A > 0, V(a;) G L°°(fi) and V(z) ^ 1. TTiera 

|it(x)| ^ Cp(p(x) and \u(x)\ ^ C Pi a, a.e. itiied, 

where c p = (p - l)(2-p)t=? and C PiA = |(2-p) (^f)^. 

Proof: By assumption, (it, 0) G #o(^) x ^o' 2 (^) * s a weak solution of (|3.ip . 
then, for any v G Hq(Q), we have 

/ V,V* + f V [x) u^ + A / - / |„ r W x = 0, (3.2, 

Jn Jn Jn 

/ V^Vt;dx= / u 2 vdx. (3.3) 
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Adding c p J n u 2 vdx on both sides of (|3.2p . and using (|3.3j) we get that 

/ \7uVvdx + / [V(x)it + CpU 2 — \u\ p ~ 1 u]vdx + A / (f>(x)uvdx 
JQ Jvl J Q (3 4) 

= Cp VcpVvdx, for any i? G i? (O). 
Based on this observation, we prove our lemma by the following two cases. 

Cases 1: O = B k . We let 

wi(x) = (u(x) — c p (j)(x)) + and Qi = {x G : 1^1(2;) > 0}. (3-5) 

then G -ffp(ri) and u(x)\q 1 ^ c p 4>(x) > 0. Taking ^(x) = in (|3,4p . 

and using F(x) ^ 1, we see that 

/ V«V^*+ / [u + ^-H'- 1 "]^^/ (3.6) 
However, for all t ^ we have t + c p t 2 - t p ^ if p G (1, 2) and c p = (p - 

2-p 

1)(2— p)p~ l . Then, (|3.6p implies that f Q VuVwidx — c p V(j}Vw\dx ^ 0, 
that is, 

V(u - c p (f))\7widx = / \Vwi\ 2 dx = 0. (3.7) 



Hence, = or wild = constant. By the definition of f^i and w\ = in 
f2\J)i, then u(x) ^ c p 4>(x) a.e. in 0,. On the other hand, replacing u by —u 
and repeating the above procedure, we see that — u(x) ^ c p cj)(x). Therefore 

\u(x)\ ^ c p (j)(x) a.e. in x G f2. (3-8) 

To prove that |n(x)| ^ C Pi a a.e. in 1 6 fl, we let 

u; 2 (x) = (u(x) - C Py \) + and £l 2 = {x & w 2 (x) > 0}, (3.9) 

then W2 G -ffo(^) and u(x)\q 2 ^ C Pi a > 0. Taking v(x) = W2{x) in (|3.2p and 
using (|3.8p and V(x) ^ 1, it follows that 

( Vu\7w2 + uw2dx ^ / |ii| p_1 iiu>2 — —u 2 W2dx ^ / C p .\W2dx, 
here we used the fact that niax{t p — -£-t 2 } = C Pt \ if p G (1, 2). This yields 

( \V(u-Cp, x ) + \ 2 +\(u-C Pt x) + \ 2 dx = ( V[u-Cp,x}Vw2+[u-Cp,x]w2dx < 0, 
Ja 2 Jn 2 
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it follows that IO2I =0, then u(x) ^ C Pt \ a.e in x G fl. Similarly, —u{x) ^ 
C Pt \. Therefore 

\u{x)\ ^ C P: \, a.e. in x G ft. (3.10) 

Case 2: Q = R 3 . In this case, it is not sure if w\(x) given by (|3.5h is in 
-?/ 1 (lR 3 ), so we need to replace w\{x) in f)3.5j) and the correspondence by 

w e (x) = (u(x) — c p 4>(x) — e) + for any e > 0. 

We claim that w t (x) G // 1 (R 3 ) for any e > 0. In fact, by the second equation 
of (13.11) and Theorem 8.17 in [T3], we know that 



|L°° 



(Bxfe)) < C (|0U«(B a d,)) + Ml«(B s (»))) > for each y e 



where C is a constant independent of y 6l 3 , This and <fi G f^ 1,2 (R 3 ) imply 

that (/>(x) ' 
(I3TD that 



that </>(x) — > 0. For any y G R 3 , taking (p G C^°(B 3 (y)), it follows from 



VuVip + b(x)mpdx = I c(x)ipdx, 

B 3 (y) ' ' JB 3 (y) 

e 

where = V(x) + A</>(x) and c(x) = \u\ p ~ u(x). Clearly, c(x) G L~i 

and b(x) G L°°(Bf (i ) for i?i > large enough since — > 0. For each 
|y| > R\ + 3, since p < 2 and | > |, Theorem 8.17 in [14J implies that 



HL°°(Bi(y)) < C Mw|jy»(B a (y)) + l c O)l L £ ( 



where C > depends only on p and \b(x)\ L ooi B c \. This implies that 

u(x) — > as |x| — > 00. (3-11) 

Hence, there is > such that suppu> e C -Br £ , w € {x) G Hq{Br e ) and 
iy e (x) G ^(R 3 ). Noting that V(u - c p 4>) = V(u - c p - e) for any e > 
0, we see that (|3.7p holds for ty e , then Poincare inequality implies that 
Id \ w e\ 2 dx < C e Jq |Vit? e | 2 dx = since w e (x) G Hq(Br € ) and ui e (x) = on 
-Bfl E \ So, ^ c p 4>(x) + e a.e. in R 3 and (|3.8p is obtained by letting 
e — > 0. By (I3.1ip . there is i?o > such that supp(ri(x) — C Pi a) + C Br and 
1^2 (x) given by ()3.9[) is in i7 1 (R 3 ), then exactly the same as in Case 1 we 
get fpUO]) . □ 

To get the L°°-norm estimate of solutions to problem (|2.ip for p G [3, 5), 
we need the following general L°°-norm estimate for functions in <^ L -' 
by using its L 2 * norm, where 2* = -^r^. 
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Lemma 3.2 Let N ^ 3, p G (1, $±§) and let u G ^ 1 ' 2 (M 3 ) \ {0} 6e a 
nonnegative function such that 

/ VuV(h(u)cp)dx < / \u\ p ~ l uh(u)ifdx, (3.12) 

JR^ JR N 

holds for any nonnegative cp G C^°(M Ar ) and any nonnegative piecewise 
smooth function h on [0, +oo) with h(0) = and h! G L°°(IR 3 ). Then, 
u G L°°(R ) ana 1 there exist C\ > and C*2 > ; which depend only on N 
and p, such that 

|u|oo < Ci (l + |n|£ 2 ) |n| 2 .. 

Proof: The main idea of the proof is Moser iterations, which is somehow 
standard. For the sake of completeness, we give a proof in the appendix 
based on p] and [Hj. □ 

Lemma 3.3 For each k G N large, let p G [3, 5) and Uk be a solution of 
problem $2.1\) given by Theorem \2.l\ Then there exist constants Mo (A) > 
and M(A) > 0, independent of k and \i, such that 

\u k (x)\ < M (A) a.e. in x G B k and \\u k \\ k + \V(pk\L 2 (B k ) ^ M(A). 

Moreover, Mo (A) and M(A) are non- decreasing in A > 0. 

Proof. By Theorem 12. H for any k G N large, saying A; > fco G N, there exist 
nfc and c\ > such that 

= CA,fe < c A and 4(«fc) = in ^t. (3.13) 
Similar to the derivation of (|2.14p . we have 

|Vn fc | i2 ( Bfc ) < \\u k \\ k ^ 2^/cJT., (3.14) 

this and (p^|) yield 

|V^li 2(Bfc) < CKI^ R , < C||«fcllfe < 16CcL, (3.15) 

where C > is independent of fc, A and pL. Let M(A) = sup {2^/c\ ;k + 

fc>fco,/i>0 

4\/C'cA ) fc}, then M(A) is finite for each A > since c kj \ ^ c\. Moreover, 
M(A) is non-decreasing in A since c\ tk is non-decreasing in A by Theorem 
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EIH Hence (gUP and (137151) imply that \\u k \\ k + \Vcf) k \ L 2 {Bk) ^ M(A). 
Define as in (|1.8f) . then uj, G and suppt^ C B k , hence (|3.14p yields 

Ifellsv ^ 2 \/c\k- (3-16) 

For any ip G Cg°(]R 3 ) and any nonnegative piecewise smooth function h with 
h! G L°°(IR 3 ) and h{0) = 0, let u = h(u+)(p, then u G % and 7j.(tx fc )u = 0, 
this yields 

VutVv + V^(x)ufvdx + A / <p k u k vdx = / | p_1, Sfc vdx. 

Since and are nonnegative, this implies that (|3.12|) holds for 

u = uf and N = 3. Then, by Lemma [3721 and f)3. 16|) there exist C\ > and 
C2 > 0, depend only on p, such that 



■u 



k l°° 



<C a (l + |«^ a )|fi+| 2 .< sup {(7i(l + cf fc )v^A,fc}:=M)(A). 



fc>fc0iA t >0 



Similarly, we know that |u fc loo ^ Mo(A). So, |u/c|L°=(_B fe ) < l^fcloo ^ Mo (A). 
Moreover, Mo (A) is nondecreasing in A > since c\± is nondecreasing in A. 
□ 



4 Proofs of Theorems 11.11 and 11.21 

For r > 0, let £ r G C°°(IR 3 ) such that 

^ = { n W I r' with |V£ r | < -. (4.1) 
u i \x\ r 

To prove our Theorems, we need the following two lemmas. 

Lemma 4.1 Assume (Gi) (G3) /10/d. Let p G (1,2), u G S'v o-nd <f> G 
L 6 (R 3 ) be such that \u\oo L and |u(x)| ^ K(j)(x) a.e. in x G R 3 , /or some 
L > and > 0. Moreover, for all rj G C]j(]R 3 ) u>i£/i 7/ ^ 0, there holds 

/ VuV(ur]) + VJx)u 2 rjdx + A / 4>u 2 r]dx = / |ii| p+1 7/dx, (4.2) 

where A > 0. Then, if fj, > ft = max{0, L p_1 — 1}, there exists > 
(depends on L, K,p, A, \x) which is decreasing in fx > p, such that 

\\ u \\@v + lj 3 <f>u 2 dx\ < M M . 
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Proof: For £ r given by (|4.ip . taking rj = £ r in (|4,2p . it gives that 

/ (|Vu| 2 +V^x)« 2 )£ r cfa:+A / 4>u 2 £ r dx = / |u| p+1 £ r <ix- / uVuV£ r dx, 

then, by \u\oo ^ L and (|4.ip we have 

/ (|Vii| 2 + (1 + fig(x))u 2 )Crdx + A / <pu 2 £ r dx 

^ L^ 1 / u 2 £ r dx + - / (|Vu| 2 + u 2 )dx, 
Jr3 r J R 3 



that is, 



/ (|Vu| 2 + [l + ixgix) - LP-^u^irdx - [ (| Vu\ 2 + u 2 )dx. (4.3) 
Jr3 ~ ' r J R 3 

Since > /Z > L p_1 — 1, for each r € there exists e T > such that 

r(l - e r ) > I^ -1 - 1. By (G 3 ) we can find R T > such that g(x) ^ 1 - e T 
for all |x| ^ i? T , then 1 + fig(x) > 1 + t(1 — e T ) > L p_1 for all |x| > R T and 
there is 5 T > such that 1 + pLg{x) > L p ~ x + 5 T for all |x| ^ R T . So, if > p, 
and r/4 > i? T , it follows from (14. 3p that, for each r £ (p-,n), 

[ (\Vu\ 2 + 5 T u 2 )dx sC - / (|V«| 2 + u 2 )dx. (4.4) 

On the other hand, taking rj = 1 in (|4.2p . it gives 

/ |Vn| 2 + (l + / u 5 (x))n 2 da;^ / If^u 2 - ^-\ufdx. (4.5) 

JR3 7R3 il 

Let /?(t) = LP~H 2 - ±t 3 (t ^ 0), then there exists C* := C{L,K,p,X) £ 
(0, +oo) such that /3(t) C* < +oo for all f ^ 0. Therefore, if r > 4i? T and 
ji> p,,it follows from (|4.4p and (|4.5p that 



(3(u)dx 



/ |Vu| 2 + (1 + fig(x))u 2 dx ^ / (3{u)dx + 

^dx + LP- 1 [ u 2 dx^C*\Br_(fd)\+L p - 1 ^- ! (\Vu\ 2 + u 2 )dx. 

Take r T > AR T large enough such that L p_1 ^- < i, then 
/ |Vu| 2 +(l + iig(x))u 2 dx s: 2C*\Bir. |. 

JR3 2 
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By Sobolev embedding, |u| p +i is also bounded above by a constant de- 
pending on C* and r. This and (|4.2p imply that there exists M T := 
M(L, K,p, A, r) such that 



2 



IMIsV + i / 4>u dx\ ^ M r , for each r € (//,//)■ 

UR3 J 

Let M M = inf M T , then M M is decreasing in fj, > fx and depends only on 

T6(/i,M) 

L, K, p, A, such that 



Lemma 4.2 //p > 1, and (Gi) (G3) /ioZg?, /ei {u k } be bounded in Sly such 
that \uk\oo ^ M (for some M > 0) and 

/ Vu k V(u kV ) + V fl {x)u 2 k r ] dx < / |u fc | p+1 r/dx, (4.6) 

/or all 77 £ C]j(IR 3 ) w't/i 77 ^ 0. Then, for each fi > max{0, M p_1 — 1}, i/iere 
exists u G smc/i i/tai, passing to a subsequence, 

\u k -u\ q ^0 for q£ [2,6). (4.7) 

Proof: Since {u/J is bounded in £Fy, there exists it G such that, passing 
to a subsequence, 

life — 1 it weakly in f^y, u k (x) — » u(x) a.e. in 2; G M 3 . (4-8) 
For £ r given by (|4.ip . taking 17 = £ r in (|4.6p , 

/ VukV(uk£ r ) +Vp(x)ul£ r dx 4: / |u fc | p+1 £ r ofe. 

Since \i > max{0,M p ~ 1 — 1}, \iik\oc ^ M and (G3) holds, similar to the 
discussion of (|4.4p . there exists > and 0^ > such that, for all R ^ 4i? /t , 
we have 

/ (|Vu fc | 2 + 5^ul)dx < ^- uniformly for k G N large, (4.9) 
J\x\>R " R 
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where T = sup ||ujt||^ v . Since H 1 (Br) L q {Bf{) is compact for 1 ^ q < 6, 
passing to a subsequence, (|4.8|) implies that 

u k (x) X u(x) inL q (B R )foil^q<6. (4.10) 
For /;£N large and any R > 4i?^ large enough, we have 

\Uk ~ U\ q ^ \u k ~ U\ Lq ( BR ) + \u k ~ «|z9(B°) 

6 — q 3g — 6 

2 \ 1 1 , ii 2 



3g— 6 / 6— g 6 — g 

< K - "U«(B K ) + C\\u k - u\\ H i(^3-) ( \ u \lI(b c r ) + I^Il2(B=) 

< K - «|w(B«) + C f Ittl^c) + (T/R)^\ by ([49]). 

By letting k -> +oo, then i? ->• +oo, we get (gTrJ). □ 

Lemma 4.3 Assume p G (1,5). If (G\) (G3) hold and {uk} is bounded in 
*2>y satisfying 

l^fcloo ^ M, for some M > 0. 

Let <fik be the solution of — A0 = u\ in Q^iBk) and cf>k be the extension of 
4>k on IR 3 defined as M.8\) such that, for all n G C B (M.' i ) with n ^ 0, 

VukS/{ukrr) + V^{x)u\ndx + \ I (pk^ndx = / \uk\ p+1 ndx, (4.11) 



where A > 0. Then, for each fi > max{0,M p 1 — 1}, there exists u G @y 
such that, passing to a subsequence, 

k k l f u ^(y) 

Iktfc — it] \® v — > and |V(<^fc — </>)|2 — > wii/i </> = — / r r dy. (4.12) 

4vr 7 R3 |x - y| 

Proof: Since A > and 0^ ^ 0, (|4.1ip implies (|4.6p . for each fi > 
max{0, M p_1 — 1} applying Lemma 14.21 we have u G @y such that, 

\u k -u\q for o G [2,6), (4.13) 
This and Lemma 12.31 imply that, passing to a subsequence, 

fc ± 1 y « 2 (y) ^ ^ K _ rt> i, 2 / m 3\ 



^>k^ <P= ~r / 1 strongly in 

47T J M 3 \x - y\ 



19 



Hence, 

/ cf) k u 2 k dx / 4>u 2 dx. (4-14) 
Jm. 3 ' Jk. 3 

Then (gjgj) follows from ([4TT31) . ([4TT41 and (l4"TTTJ) with 77 = 1. □ 

Now, we are ready to prove Theorems 11,11 and 11.21 For u k £ S^k, 4> k & 
S>Q' 2 {B k ) and a > 0, c\ > given by Theorem 12.14 1st tt^ G f^y and 0fc G 
£F 1,2 (K 3 ) be the extensions of u k and 0& on R 3 defined as (jl.8p respectively. 
Then, 



^/ |Vu fc | 2 + V^(x)^dx + - / <j) k (x)u 2 k dx ^— f \u k \ p+1 dx = c Afe , 

2Jr3 4 J r3 p+17 R 3 

(4.15) 

and ca,^ G [«,ca]. Moreover, for any if G i§V with supp</? C B k , by I' k {u k ) = 
we see that 

/ Vu k Vip + V fJi (x)u k (pdx + A / cj) k u k ipdx = I \u k \ p ~ 1 u k ipdx. (4.16) 

JR3 ' 7R3 JR3 

Note that for any tp G Co°(M 3 ), suppy? C B k as k — > +00, therefore, 

Vu k Vip + V tM (x)it k (pdx + A / (f) k u k (pdx = / |%| p ~ 1 %v9(ix + o(l). 

Jr3 Jr3 

(4.17) 

Proof of Theorem ll.lt Let A* be given by Theorem 12.11 and A* < +00 

if p G (1,2). For A G (0, A*), let (u k ,(p k ) £ & k x &l ,2 (B k ) be a solution of 



(|2.ip for each A; G N large. Applying Lemma 13.11 with £1 = B k , there exist 
c p > and C Pj a > such that 

\u k (x)\ ^ c p 4> k (x) and |u fc (z)| ^ C P)X , a.e. in x G B k . 

Then, 

|%(^)| ^ Cp^k{x) and ('Ufc(x)l ^ C Pi A) a -e. in x G M 3 . 

Since u k r) G ^ for any 77 G C^(M 3 ), it follows from I' k (u k ) = that (f4T2|) 
holds for u = u k and 4> = <fi k , applying Lemma I4TT1 with L = C Pj a, K = c p 
and 7X = Tjfc, there is M M := M(p,\,fi) > 0, which is decreasing in 77 > 
such that 



\u>k\\®v + |V0fe|2 = \\uk\\» v + t I (p k u 2 k dx\ ^ M M . 
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Similarly, {%} and {<pk} satisfy (|4.1ip by I' k (uk) = 0. Hence, Lemma POI 
shows that there exists u £ such that, passing to a subsequence 

Uk — > u strongly in an d <^fc ~~ * 4>u strongly in f^ 1,2 (]R 3 ). (4.18) 
Furthermore, 

hhv + IWu|2 < M M and |n|oo < C p>A . (4.19) 

Combining (|4.15p and (|4.18p . we have I(u) £ [a, c\]. On the other hand, by 
fljTTD , for any <^ G C^QR 3 ), as fe -> +oo, I'(u k )<P = then > usm S d4TT8|) 
we see that I'(u)(p = 0. So, u is a nontrivial solution of (jl.ip in SV. By the 
second equation of (jl.ip and Theorem 8.17 in p3], for any y £ R 3 , we have 

sup W < C(|0u|L«(Ba(tf)) + M£«(B 4 (»)))> 
Bi(l/) 

where C > is a constant independent of u and y £ M 3 . This and (|4,19p 
show that 

sup u (x) < CM„(1 + M M ) 4 C(p, A, /x) =: C M , (4.20) 

ireIR 3 

where C M > is a constant dependents only on p, A, fi. □ 

Proof of Theorem [L2} For p G [3,5) and A > 0, let (u k ,<j) k ) £ % x 

^ 1,2 (i?fc) be a solution given by Theorem 12. 1[ By Lemma I3T31 we know that 

\uk\oo < M (A), + |V(/>fc| L 2 (jBfe ) < M(A), 

where Mo (A) and M(A) are given by Lemma 13.31 Hence, 

|-Ufc|oo ^ M (A), ||«fe[|^ + |V^fe| 2 ^ M(A). 
Applying Lemma 14.31 with u& = %, there exists u £ S>v such that, 

u k ^u in @y. (4.21) 

Moreover, 

Nla^ + |V^u| a < M(A) and |u|oo < M (A). (4.22) 

Hence, ([4.2ip and (|4.15p show that I(u) £ [a,c\], where a and c\ are 
constants given by Theorem 12.11 On the other hand, by (|4.17|) . for any 
if £ C^(R 3 ), as k -> +oo, I'(uk)v = o(l), and i* follows from (IOTP that 
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I'{u)(p = 0. So, u is a nontrivial solution of (jl.ip in 3>y. Noting (|4.22|) . 
similar to the discussion of (|4.20p . we have 



sup <j) u (x) < CM(A)(1 + M(A)) =: Mi (A), 

zgR 3 

where Mi (A) is a constant and independent of \i. By Lemma 13.31 M(A) is 
non-decreasing in A > 0, hence Mi (A) is also non-decreasing in A. □ 



5 Proofs of Theorems Q to [L5] 

Proof of Theorem II. 3t By Theorem 11.21 if u > fJ^(X), problem (jl.ip has 
always a solution u\ G f° r an Y A G (0, 1) such that 

KII^ + |V0 U J 2 <M(A) <M(1) and |n A |oo < M (A) < M (l), 

since Mo(A) and M(A) are nondecreasing in A > 0. Then, there exists 
uq G S>v-, passing to a subsequence, such that 

u\ uq weakly in Q)y and u\ Uo, a.e. in x G M 3 . (5.1) 
For any ip G @ v , L 3 (j) ux u x ydx ^ |0 Ua | 6 |«a| 12 Mi2 < C(M(1)) and 

5 5 

/ Vu\V(p + V^(x)u\ipdx + \ / 4> Ux u\Lpdx = / |uA| p ~ 1 UA^dx, (5.2) 
Jr 3 Jr 3 Jr 3 

let A — > 0, we have 

VuqV<£ + V^{x)uQipdx = l \uo\ p ~ 1 uoipdx, (5-3) 



that is, uo is a weak solution of (jl.ip with A = 0. Since uyq G Sly for any 
r/ G C^(M 3 ) with ry ^ 0, by (JS^D and A > 0, we see that 



By Lemma [OJ there is n* G *3y such that tja u* in L 9 (IR 3 ) for g G [2, 6). 
Hence u* = no by (|5,ip . Let 99 = in (|5.2p and 99 = no in (|5.3p . since 
/k3 <t>ux u \dx = |V0uxll ^ M 2 (l), as A — ► 0, we have ua — > no in fIV- This 
and I(u\) G [a, c\] for A > imply that Io( u o) ^ a > 0, hence no ^ 0. □ 
Proof of Theorem 11.41 To make it clear that 3>y depends on «, here 
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we denote 9y by Q^. Let > ma:K {Cp~\ — 1,Mq _1 — 1} and > 
by Remark 11,11 For \i (iq , let be the solution given by Theorem 11.11 
or Theorem 11.21 then {u^} is bounded in D M0 since < M M0 . For some 
u G Z&fjQ C i? 1 (M 3 ) we may assume that 

Up — u weakly in f^ Mo and — > -u a.e. in x G M 3 , as /i — > +oo. (5.4) 

Then, Fatou's lemma shows that 

f f 1 M 2 

/ g(x)u 2 dx < lim / g(x)u 2 u dx < lim — lltijl® ^ lim — — = 0, 

so (Gi) and (G2) means that w(x) = a.e. in 1 £ I 3 \ f^- O n the other 
hand, for any ip G by I'(up) = we have 

/ Vu At V<£ + (l + ug(x))upipdx + \ I <f> u u ti <pdx= / 

(5.5) 

For each fi ^ hq and 77 G Cjj(lR 3 ) with 77 ^ 0, it follows from (|5.5|) and (Gi) 
that 



/ VupViupTj) + (1 + fi g(x))u 2 r]dx sC / 

JR 3 JR E 



Un\ p+1 r]dx. 



Then, it follows from (|5.4p . Theorem 11.11 or 1 1.21 and Lemma 14.21 with Q!y = 
and {u k } = {u^ that 

K - u|/^t°° for qe [2,6), (5.6) 
hence, Lemma 12.21 yields that 

0unU 2 u (x)dx M / <j)uU 2 {x)dx. (5.7) 

Let 93 G Cq°(1R 3 ), by dS3D and dSZZD (see (3.18) in [TS] for the details), we 
have 

/ 4'u t ,u fl (x)ip(x)dx ^^t 00 / (f) U u(x)ip(x)dx, (5.8) 
Jr 3 JR3 

and dHHJ) holds also for tp G tf^R 3 ) by the density of Cg°(M 3 ) in H 1 ^ 3 ). 
Let <p = u in (|5.5|) and in (|5.8|) . then, by (|5.4|) and (|5.6|) we have 

f \Vu\ 2 + u 2 dx + A / (feu 2 dx = / (5.9) 
Jr 3 jr 3 jr 3 



23 



By ()5.6p and (|5.5p with tp = u^, we see that 
/ \Vu^\ 2 + uldx + \ f <p u u 2 (x)dx^ [ \ Ut ,\ p+1 dx^°° [ \u\ p+1 dx. 

JR3 * J R 3 * J R 3 J R 3 

(5.10) 

It follows from ([SUjl and ([STTD]) that 

lim / | VuJ 2 +« 2 <ix+A / < fiu u v 2 u dx ^ / | Vn| 2 +?2 2 (ix + A / (f>uU 2 dx, 

this and (|5.7p show that 

lim / |ViiJ +u u dx^ / V« + u cte, 

and the lower semi-continuity of norm implies that 

u^^u, ini^IR 3 ) as fi -> +00. (5.11) 

Now, we claim that u(x) ^ on Oo- Otherwise, u = a.e. in x E M 3 , 
then |it M | p+ i M _tf° and J <j) u ^u 2 dx ^—t^ q by (|5.7j) . hence /(u^) — > 
as fjL — > +00, which contradicts the fact that I(u^) ^ a > 0. Since cft7o is 
Lipschitz continuous, we have u 6 //^(f^o) by Ct(x) = 0, a.e. in M 3 \ Qq. It 
follows from (jBTTTj) . (pT8|) and ([S3]) that, for any 93 € #0(^0), 

^ ^ , A f u 2 (y)u(x)(p(x) . . f 1 

VuVip + uc^cte H / — , v , ' dydx= / |n| p_1 u^dx, 

4?r jQ oX n F - y| 7n 

that is u is a weak solution of (|1.12p . □ 

Proof of Theorem II. 51 This proof is motivated by that of Theorem 1.3 
in 0. Let (i Q = max{3(-Mj -1 - 1),?>{C P ~^ - 1)}. By Theorem O or 
the L°° -norms of ix^ eliicI (fouu 

are bounded uniformly in /j, ^ (j,q. By (G3), 
there exists Rq > such that ^(x) > | for |x| ^ Rq. Then, we have 

(-A + ii)u 2 = -2(W^ - |)u 2 - 2|Vn M | 2 < for \x\ > R and \i > /x , 

(5.12) 

where W^(x) = l + /j,g(x) +X(j) u (x) — \u^(x)\ p . On the other hand, let 
be the fundamental solution of — A + /x, then, it follows from Proposition 3.1 
and Theorem 4.2 in [8j that is C°° outside the origin and it is positive 
such that 

(-A + i^Wn = for |x| > Rq and (5.13) 

-(iV-l) _ I 

= \x\ 2 e vMFl^ + o(l)) as \x\ — ► +00, uniformly in /x ^ /Jq- (5.14) 
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Let C = max{M , C PiA }, A ^ Wti ° Ro) and v(x) = u^(x) - Aw^. For /x > /x , 
it follows from (I5TT2D and ([5T3]) that 

f {-A + n)v < 0, |z| > i?o, 

I D ^ 0, |x| = iio- 

The maximum principle (Theorem 8.1 in p3]) implies that 

uAx) ^ Aw fl (x) for |x| > i?o- 
By (|5.14p . there exists A > independent of /x > /xo such that 

u M (x) < ^Ix^^-^d^l-^o) f or ■> ^ Q anc j ^ > ^ . □ 

6 Ground state of ( TTTTT ) for p £ (1,2) 

Proposition 6.1 lei jY = {u G & v \ {0} : !'(«) = in S>y} and A* 
be given by Theorem li.il C p ^\ be given by Lemma I3.il Then, for each 
Ag(0,A*) and /x > Cj" 1 - 1, 

(i) c/K 0, and f/iere exists C p > suc/i iTiai ||n||^ ^ C p /or all u G JV . 

(ii) There exists C\ jfl > such that sup (||n||^ v + |V0 U |2) ^ Ca,/z- 

(iii) «/f C f^y *s a compact set. 

Proof, (i). Since A G (0, A*) and /x > — 1, Theorem 11.11 implies that 

=yf ^ 0. If u G JV , then I'(u) = and u ^ 0, that is, 

/ \Vu\ 2 + V tl u 2 dx + X f u u 2 dx= [ \u\ p+1 dx. 

Hence, ||n|||y ^ f R3 \u\ p+l dx ^ 5' p+1 ||n||^ 1 and ||n||^y ^ S ~ := C p . 
(ii). Since || • \\@ v and || • ||#i(R3) are equivalent norms in S>y, by Lemma 
13.11 with = R 3 , we have that sup |u|oo ^ C Pt \ and \u(x)\ ^ c p 4> u {x) a.e. 

in x G R 3 . For u G =yf , /' (n) = and (|4.2j) holds. Then Lemma 14.11 implies 
that 

IMbv + IW«|2 = + {/ 3 ^'" 2da; | < Ca./x' for some Ca, m > 0. 
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(iii). Let {u n } C <jV , then I'(u n ) = and (|4.6p holds for u n . By part (ii) 
and Lemma 14.21 there exists u £ £Fy such that u n ^ u strongly in L 9 (R 3 ). 
This and Lemma 12.21 show that <p Un A strongly in ^ 1,2 (R 3 ). Thus, for 
any </> G i§V> 

/ Vu n Vip + V^u n (pdx A / VuV(p + V^iupdx, 
/ (t'unUnfdx A / (f> u u(pdx and / |n n | p_1 n ra <^dx A / |ii| p ~ 1 uc / 2(ix. 

J]R3 J R 3 J R 3 J K 3 

It follows from I'(u n ) = that = in S>y. Then, using I'(u n )u n = 

I'(u)u we know that u n —> u strongly in @y, this and (i) show that ||it||^ v ^ 
C p , hence u £ JV and .jV is a compact set in S>v D 

Proof of Theorem 11.61 By Proposition 16.11 (iii), we know that / is 
bounded below in ,jV and cq = inf I(u) > — oo, then there exists {u n } C j¥ 

such that I(u n ) — > cq. So, Proposition 16.11 (ii), (iii) imply that, there exists 
uo £ ,jV such that I(uq) = Co and I'(uq) = 0, that is, uq is a ground state 
of ([LTD. □ 



7 Appendix 



Proof of Lemma 13.21 For /3 > 1, M > 1, the same as [19] , we define a function 
#6 C^O.+oo) by 

s' 3 , se[0,M], 
/3M l3 - 1 s - 03 - 1)M' 3 , s € (M, oo). 



Let 

G(s)4 j*\H'(t)\ 2 dt = j ~ 



0*M*V-»s - ^lfs^ M^~\s e (AT, oo). 



Then G(s) and H(s) are Lipschitz in [0, +oo), therefore, G(u) and i?(u) are in ^ 1 ' 2 (R JV ) 
if U 6 ^ 1,2 (R JV ). Moreover, 

sG(s) < s 2 H' 2 (s) < l3 2 H 2 {s). (7.1) 
Let 77(35) G Co°(R N ) with 77(35) ^ such that, for ri > r2 > 0( n will be determined later) 

2 

77(35) = 1 for x G B T2 , 77(33) = for x G , and |Vr/| ^ -. 

for each y G R^, setting r](x) = 77(35 - j/) 3s and < J? 2 G(u) G ^ 1 ' 2 (R JV ) with compact 
support in B ri (y) n {3; G R 3 : u(x) 0}. By (J7TTJ) and (f3TT2]l with 95 = rj 2 and h = G, 

/ VuV(rj 2 G(u))dx < / u p_1 W77 2 G(M)da: ^ /3 2 / |u| p ~V i/ 2 (it)d:c. (7.2) 

Jk" Jk" 



26 



Noting that 

VuX7(r] 2 G(u)) = \Vu\ z H"{u)rj z + 2VuVr]G{u) V . 
Also, by Young inequality and (|7.1|l , 

\VuVrjG(u)ri\ = |??u~ 1/2 G 1/2 (u)Vu||u 1/2 G 1/2 (u)V?7| 



^ -\Vu\ 2 H' (u) V 2 + Af3 2 \Vr]\ 2 H 2 (u). 



Then, it follows from (17.211 that 



/ \S7u\ 2 \H'(u)\ 2 ri 2 dx < 16/3 2 ( / \Vri\ 2 H 2 (u)dx+ [ |u| p_ V# 2 («)* 
Hence, by /3 > 1 and Holder inequality we see that 

\V{H(u)r])\ 2 dx ^2 f \Vr)\ 2 H 2 (u)dx + 2 f \ Vu\ 2 \H'(u) \ 2 rj 2 dx 



■ 2 



(f \Vrj\ 2 H 2 {u)dx+ [ \u\ p - 1 rj 2 H 2 (u)dx) (7.3) 

sS 34/3 2 ( f \V V \ 2 H 2 {u)dx+\ V H(u)\ 2 2 *\u\ p 2 7 1 \B ri \' 1 ) , 

where 7 = 2 g? -1 an d l-Sn I denotes the volume of B ri . Taking j3 = ^- = j3o and 

rx = (68/ag|«|Sr l + l)~* <min{|Bir^,|Bi|-* (68^ H^ 1 )"*} . (7.4) 

Obviously, 34/3 2 lug" 1 |B n I 7 < § and |B n | ^ 1. Then, with /? = /3 gives that 



/ 

■Jr' 



\V(H(u)v)\ 2 dx ^ 68/3 2 / IV^-H^dx. 



Hence, by Sobolev inequality and rjH(u) £ 3)\' 2 (B ri (y)) as well as the definition of 7], we 
see that 

< f/ |H(«)»/| 2 '«fc) 7 <V/ |V(fr(u)ij)| 2 d!i! 

< 68S ( 7 1 A) / \Vv\ 2 H 2 (u)dx 

Jm N 

< (;^)V(«)lWv)>> ( 7 - 5 ) 



where C = y 68S 1 and So is the Sobolev constant, which depends only on AT. By 

definition of H(s) = s 13 if M — * +00. Noting that (3 = j3o and 2/3o = 2*, it follows from 
(1731) that 



M ^c ^ ( M fo ( 7 . 6) 

For each i ^ 2, let r, ; = 2+2 - n ( n is given by (EH). Take fji G Cq°(R 3 ) such that 

2 

nifsc) = 1 for x e B r . +1 , ee for x 6 \ B r and IV^I < . 

n — n+i 
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Let S = 2g ll\_ p and S € (0, 1) by p G (1, f±§). For i > 2, applying O with r, = 

rji(x) = fji(x — y) and = /3i = <5 _l > 1, then noting that |S ri | < |S ri | ^ 1 and using 
Holder inequality and the definition of fji, we get 



| V(# | 2 dx ^ 34ft 2 f f \V Vl \ 2 H 2 (u)dx + [ K" 1 rj 2 H 2 (u)dx) 
\Jm N ii» / 



< 34/3 2 



< 34/3 2 



n — ri+i 



|-ff(«)l£>»*(B P4 (»)) + \ H ( U )\l^HB T .(y))H T , 



p-1 

i^(Br( («)) 



i 2/5 o(Sr 2 (y)) 



|ff(u) 



L 2 *HB r (y)) 



+ 



Ti — r i+ i J \T\—T2 



(p-l)//3o 



|P-l 

U 2 *(-B ri (!/)) 



\H(u)\^ HBr . m by (76 



r« - n+i 



£=*«(S ri (y))l 



where C is a constant depending only on JV and p. Since rjiH{u) € £^g' 2 (.B ri (y)), it follows 
from the Sobolev inequality that 



\H{u)\%* {B , v)) < S-,7 1 / \V(H(u) Vi )\ 2 dx < 



\H{u) 



L 2 * 5 (B r - (y)) 



where C\ = C\/ Sg 1 depends only on N and p. Let M — > +oo, and -ff(w) = it /3 % then 



l w L 3 *ft(B n+1 {s,)) < 



+ 1«| 2 ^ 1)/2 ) 



I u Il 2 *".-i( Bi . i(H ))- 



n — ri+i 

We can now perform the Moser iterations in a standard way and get that 



\ u \l 2 '^ (b {y)) < n 



n - n+i 



u \l 2 ^ H b {v)) (7.7) 



2, /(< ^8C 1 (l + Hr i)/a ) VW 



n 



\ U \l 2 ^HBr 2 (y)y 



where /(i) = £ + + ^ - and ffC fl = - A- Since 

2*/3i = 2/3q + 1 - P < 2/3 2 , by Holder inequality, (73) and (176) . we have 



t-Br-JP- 1 l«l 



< C/3q 

£ 2/3 o(Sr 2 (!/)) ^ |M| i 2 ^(i3 r2 (!/)) ^ Vri-r 2 



11 2* 



This and (17.711 show that 



''i 



ri 



u\ 2 * 
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Let i — > +00, and noting (I7.4p . There exist positive constant Ci(p, N), Ci{p, N) such that 

2^Mi_£ / 8 C 1 (l + j M |^r 1)/2 )\ T ^ ffc^ 1/0 ° 



2 Mi^/ 8c l( i + H r^) y-7 
l w U~(s, l/2 («)) < (5) (1 5) 1 1 ^ 



''1 



^ 2^ (8C 1 + ^o)(l + H2 -r) 
£ I ri / 



25 2 -5 3 / 1 _ Ifi p-l t \ TT^S' + '^TT 

< (|)^^ ^|Bi|-(8C 1 + ^C/3o)(l + h| 2 . 2 )(68/3 >|^ 1 + l)"J |u| 2 , 

g 1 (p,jv)(i + MS ,(,, ' ,0 )M2.. 

Since y € and ri is fixed by (|7.4[) . we have 

|u|» *ZCi(p,N) (l + |w|£ 2(p ' N) ) |«| a .. □ 
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